Skip to content

Conversation

@jessegrabowski
Copy link
Member

Closes #454

This PR lets users use the basinhopping algorithm to fit PyMC models. This is in the class of simulated annealing algorithms, except that it also runs an inner optimization loop to find the nearest minimum after proposing a new starting position.

API is pretty clean, if I do say so myself:

    with pm.Model() as m:
        mu = pm.Normal("mu")
        sigma = pm.Exponential("sigma", 1)
        pm.Normal("y_hat", mu=mu, sigma=sigma, observed=rng.normal(loc=3, scale=1.5, size=100))

        optimized_point = find_MAP(
            method="basinhopping",
            use_grad=True,
            use_hess=False,
            use_hessp=True,
            progressbar=True,
            gradient_backend="pytensor",
            compile_kwargs={"mode": "JAX"},
            minimizer_kwargs=dict(method="trust-ncg"),
        )

Gives:

  Process                            Elapsed   Iteration   Step    Target   Objective     ||grad|| 
 ──────────────────────────────────────────────────────────────────────────────────────────────────
  Basinhopping   ━━━━━━━━━━━━━━━━━   0:00:02   100/100     0.617   0.500    186.98814     0.00000
  Minimize       ━━━━━━━━━━━━━━━━━   0:00:00   8/8         0.000   0.000    34964.56397   0.00000

The top progress bar shows the overall algorithm, the current step, the target step size, current best objective function, and the gradient at that current best. The bottom shows progress on the inner optimization loop, so it gets updated a lot. You can see here that the final inner optimization went somewhere stupid, so the objective value is very high.

@jessegrabowski
Copy link
Member Author

I don't think the CI failure is our fault here, seems like a dask bug. I opened an issue on their repo.

@jessegrabowski jessegrabowski added the enhancements New feature or request label May 2, 2025
Copy link
Contributor

@AlexAndorra AlexAndorra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super neat, thanks @jessegrabowski ! Should I merge despite the CI failure then?

@jessegrabowski
Copy link
Member Author

I'm going to try version pinning dask first

@jessegrabowski jessegrabowski merged commit f2656f5 into pymc-devs:main May 2, 2025
16 checks passed
@jessegrabowski jessegrabowski deleted the basinhopping-map branch May 2, 2025 23:13
andreacate pushed a commit to andreacate/pymc-extras that referenced this pull request Jul 22, 2025
andreacate pushed a commit to andreacate/pymc-extras that referenced this pull request Jul 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancements New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add basinhopping to allowed methods in find_MAP and fit_laplace

2 participants